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Abstract 

The  study  and  practice  of  human  reasoning ,  has  undergone  significant  change  in  recent 
years  from  both  a  theoretical  and  methodological  perspective  [1J.  With  the  advent  of 
ubiquitous  communication,  advances  in  computing  and  numerous  available  analytic 
support  tools,  the  push  for  instantaneous  reporting  and  analysis  has  become  the 
expectation  [2] .  This  is  particularly  prevalent  in  the  intelligence  community  (IC)  and  in 
the  military,  where  commanders  (and  their  leadership)  expect  not  only  a  bird’s-eye  view 
of  operations  as  they  happen,  but  a  play-by-play  analysis  of  operational  effectiveness. 

While  we  have  seen  rapid  technological  advances  to  support  collections  analysis  and 
dissemination;  it  is  unclear  if  our  human  analysts  made  similar  advances  or  whether  they 
have  even  benefitted  from  the  technological  advances.  Are  human  analysts  capable  of 
making  instantaneous  assessments  or  are  they  simply  restating  what  the 
sensor/computing  “black-box”  is  spitting  out?  This  paper  explores  the  use  of  the  analytic 
decision  game  (ADG)  as  a  method  to  train,  not  only  the  human  analyst,  but  the  data 
fusion  systems  that  inform  much  of  the  black-box  technology >  being  touted  as  the  next 
intelligence  solution.  The  ADG  is  an  adaptation  of  a  proven  training  and  planning 
methodology >  -  the  military  war  game.  It  presents  students  with  a  set  of  realistic  problems 
and  challenges  them  to  work  toward  actionable  intelligence,  course  of  action  or  probable 
solution.  Such  training  enhances  the  capability  of  the  human-in-the-loop  in  a  data  fusion 
system  (viz.,  the  “human  fuser”)  as  well  as  providing  a  basis  for  training  and  testing  on 
the  same  data  sets.  The  ADG  can  also  be  used  as  a  database  for  the  test  and  evaluation 
of  algorithms  and  the  data  fusion  systems  they  support.  The  ADG  is  not  the  solution;  but 
the  pathway.  This  paper  describes  the  concept  of  ADG  and  describes  both  research  and 
training  being  conducted  at  Penn  State  using  new  multi-source  synthetic  data  sets. 
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sensor/computing  black-box  is  spitting  out?  This  paper  explores  the  use  of  the  analytic  decision  game 
(ADG)  as  a  method  to  train,  not  only  the  human  analyst,  but  the  data  fusion  systems  that  inform  much  of 
the  black-box  technology  being  touted  as  the  next  intelligence  solution.  The  ADG  is  an  adaptation  of  a 
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The  military  community  has  made  extensive  use  of  simulations  across  the  entire  spectrum  of  warfare  - 
strategic,  operational  and  tactical.  The  U.S.  Department  of  Defense  (DOD)  has  led  this  effort  since  the 
1950s;  however  the  notion  of  the  military  war  game  can  be  traced  back  to  the  early  19th  century.  The 
modem  war  game  is  attributed  to  Prussian  military  tactician,  Georg  Leopold  von  Reisswitz’s.  Reisswitz’s 
Kriegspiel  [3]  introduced  the  elements  of  force  on  force  engagements,  the  use  of  tactical  terrain,  game 
umpires  and  the  establishment  of  game  rules.  The  Prussian  military  introduced  war  gaming  in  their  officer 
training  in  the  1820’s;  the  U.S.  military  adopted  the  practice  in  the  post-Civil  war  era  for  use  at  the 
military  academies  in  the  1880’s. 

The  2005  edition  of  Joint  Publication  1-02,  DoD  Dictionary >  of  Military ?  and  Associated  Terms  defines 

war  game  as  a  “simulation . of  a  military  operation  involving  two  or  more  opposing  forces,  using 

rules,  data  and  procedures  designed  to  depict  an  actual  or  assumed  real  life  situation”  [4],  The  term  war 
game  is  no  longer  part  of  official  the  DoD  lexicon,  with  no  listing  in  the  most  recent  edition  of  Joint  Pub 
1-02  (2012).  The  term  and  concept  has  been  modernized  by  the  system,  “modeling  and  simulation”  [5]. 
Whereas  DoD  defines  model  as,  “A  physical,  mathematical,  or  otherwise  logical  representation  of  a 
system,  entity,  phenomenon,  or  process;”  and  simulation  as,  “...a  method  for  implementing  a  model  over 
time”  [6].  Even  though  the  terminology  has  changed,  the  notion  of  the  war  game  is  actively  used  in 
planning,  analysis  and  training.  In  particular  it  is  being  integrated  into  traditional  education  pedagogy  at 
places  like  The  Pennsylvania  State  University,  where  the  analytic  decision  game  (ADG)  has  become  a 
staple  for  training  future  analysts  of  security  and  risk  and  for  use  in  the  test  and  evaluation  of  data  fusion 
processes. 

The  ADG  is  an  adaptation  of  the  Tactical  Decision  Game  (TDG);  similar  to  military  war-game 
simulations  popularized  by  military  planners  and  tacticians  to  increase  the  understanding  of  tactical 
awareness  and  decision-making  [7].  The  ADG  was  adapted  by  Graham  [8]  to  promote  exercise-based 
instruction  for  teaching  and  practicing  structured  analytic  techniques,  decision-making  and  critical 
thinking  for  the  Security  and  Risk  Analysis  curriculum  in  the  College  of  Information  Sciences  and 
Technology  at  The  Pennsylvania  State  University.  Several  large  scale  decision  games  have  been 
constructed  for  this  purpose;  each  will  be  described  briefly  herein.  Whereas  the  TDG  focuses  at  the 
tactical  level  of  decision-making,  the  ADG  adopts  an  operational  (or  strategic)  view  of  the  engagement 
space.  The  ADG  does  not  exercise  military  or  law  enforcement  tactics.  However,  a  purposefully 
constructed  ADG  could  be  used  for  such  a  purpose. 

In  the  threat/risk  management  process,  for  military  operations,  law  enforcement  or  emergency 
management,  the  testing  and  evaluation  of  plans,  operational  concepts,  command  and  control  procedures 
or  incident  management  strategies  is  critical.  Simulations  can  be  invaluable  for  developing  and  evaluating 
such  processes.  Simulations  can  also  expose  biases  emerging  from  group  think;  identify  gaps  and 
vulnerabilities;  and  highlighting  areas  for  improving  internal  processes  and  inter-organizational 
cooperation.  The  value  of  using  simulation  and  modeling  to  assess  real  problems  should  be  a  critical  part 
of  any  security  plan  or  evaluation  of  operational  processes.  While  modeling  and  simulation  benefits  from 
the  participation  of  subject  matter  expertise,  innovations  often  emerge  from  novice  participants  who  are 
not  blinded  by  pre-conceived  notions  or  embedded  processes. 
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This  notion  is  especially  true  in  the  examination  of  security  and  risk  issues  that  are  increasingly 
influenced  by  emerging  technologies  and  social  media.  The  current  generation  of  students  -  the  so-called 
digital  natives  [9]  outfitted  with  a  mobile  suite  of  sensors  and  acting  as  both  sensing  and  reporting 
platform,  are  developing  novel  ways  to  communicate,  collaborate  and  analyze  data.  Participatory  sensing 
has  been  described  as  “the  citizen-powered  approach  to  illuminating  the  patterns  that  shape  the  world...” 
[10].  Students  acting  as  intelligence  analysts  are  adapting  analytic  methods  to  fit  their  modes  and  methods 
of  social  communication  with  very  positive  results.  To  look  at  today’s  student  participating  in  an  analytic 
decision  game  is  to  get  a  glimpse  at  tomorrow’s  analyst;  they  are  building  their  own  set  of  digital  tactics, 
techniques  and  procedures  (digital  TTPs)  and  applying  them  to  real-life  problems.  Again,  the  ADG  is  not 
the  solution,  but  the  pathway. 

The  ADG  is  a  scenario-based  simulation  involving  synthetic  data  elements  (reports,  observations  and 
intelligence  products)  that,  when  considered  from  end-to-end  tell  a  story  or  suggest  a  probable  course  of 
action  [11].  While  the  scenario  may  resemble  actual  events,  by  and  large  the  dataset  supporting  the  ADG 
is  not  real.  It  is,  however,  intended  to  be  representative  of  the  types  of  information  that  are  observed  and 
reported  within  the  analytic  environment  represented.  The  ADG  scenario  is  not  intended  to  represent 
actual  military,  intelligence  or  law  enforcement  methods,  intelligence  collection  efforts,  intelligence 
systems  or  decision  support  mechanisms. 
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Figure  1:  The  Simulation  Continuum  [from  12] 

Figure  1  depicts  the  simulation  continuum  indicating  the  utility  of  such  tools  that  can  be  scaled  and 
focused  to  suit  a  wide  range  of  training  participants  and  objectives.  The  simulation  offers  a  simple, 
adaptable  and  effective  method  of  providing  realistic  training  in  a  time  and  resource-constrained 
environment.  Taylor  portrays  the  simulation  continuum  as  one  that  extends  from  field  exercises  through 
computer  simulations  to  analytic  models,  where  the  realism  of  forces  on  the  ground  (live  maneuver)  is 
countered  by  the  economy  of  abstract  simulations  [12].  As  Schmitt  states,  “...there  is  no  substitute  for 
experience  of  the  real  thing,  it  can  be  hard  to  come  by  and  tragically  unforgiving”  [7].  Simulations  offer  a 
practical  alternative  that  is  reusable  and  tailored  to  the  specific  needs  of  the  user.  That  need  extends 
beyond  the  exercise  environment  and  into  the  area  of  test  and  evaluation. 

1.1  ADG  for  test  and  evaluation 

There  is  a  rising  demand  for  new  and  improved  data  fusion  techniques  and  algorithms  that  can  address  the 
evaluation  and  processing  of  hard  and  soft  data  types  [13].  Much  progress  has  been  made  in  recent  years 
in  refining  hard  sensor  processing,  mostly  in  the  area  of  one-dimensional  signals  [14].  More  recently  the 
research  focus  has  shifted  to  include  logical,  non-signals  based  data  with  more  emphasis  placed  on 
developing  natural  language  processing  techniques  and  software.  Emerging  research  supporting  the 
integration  of  hard  and  soft  data  types  into  a  common  analytic  framework  poses  a  myriad  of  problems,  not 
the  least  of  which  is  the  supporting  data. 
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One  major  limiting  feature  in  the  research  of  integrated  hard  and  soft  data  fusion  is  the  availability  of 
realistic  unclassified  data  sets.  This  is  particularly  true  for  data  that  deals  with  the  challenges  of  complex 
operations,  such  as  counterinsurgency,  terrorism  or  other  areas  of  security  and  risk.  One  means  to  address 
this  shortfall  is  through  the  use  of  analogous  scenarios  and  corresponding  datasets.  To  fill  the  void 
researchers  at  The  Pennsylvania  State  University  developed  a  synthetic  dataset  called  SYNCOIN  [15]. 
SYNCOIN,  which  stands  for  the  SYNthetic  COunterINsurgency  (SYNCOIN)  data  set  was  created  to 
support  the  development  and  implementation  of  hard/soft  fusion  analysis  processes.  Developed  at  Penn 
State  and  partially  funded  under  a  grant  by  the  U.S.  Army  Research  Office  (ARO),  SYNCOIN  filled  the 
need  for  realistic  representational  data  [16],  [17]. 

The  focus  of  the  SYNCOIN  counter-insurgency  (COIN)  dataset  is  to  support  the  development  and 
implementation  of  fusion  and  analysis  processes,  data  stores  design,  and  the  development  of  process 
flows  and  interfaces  required  for  both  automated  processing  and  user  in-the-loop  analysis  to  enhance 
Situation  Awareness  (SA)  and  Situation  Understanding  (SU)  [18].  It  was  developed  specifically  for  a 
multi-year  Multidisciplinary  University  Research  Initiative  (MURI),  sponsored  by  the  Army  Research 
Office.  SYNCOIN  focuses  on  the  intricacies  of  improvised  explosive  device  (IED)  networks  and 
operations.  The  scenario  takes  place  in  the  waning  months  of  Operation  Iraqi  Freedom  (OIF)  and  is 
largely  centered  in  Baghdad,  Iraq  [19]. 

The  data  models  considered  for  the  development  of  the  SYNCOIN  dataset  are  intended  to  mimic  the 
analysis  of  various  unstructured  messages  derived  from  the  tactical  intelligence  effort  within  the  counter¬ 
insurgency  domain.  The  messages  are  intended  to  support  the  test  and  evaluation  of  entity  extraction  and 
semantic  reasoning  techniques  to  support  entity  capture  for  various  fusion  model  elements  emulated 
within  the  SYNCOIN  dataset,  including: 

•  Agent 

o  Person :  High  Value  Target  (HVT),  Person  of  Interest  (POI),  Group  header,  Interlocutor, 
Merchant,  Informant,  Victim,  Witness,  Terrorist,  Detainee,  Police,  Military,  Ethno-Religious, 
Political,  Foreign  agent,  Insurgent 

o  Organization:  Militia,  Military,  Police,  Terrorist,  Ethno-Religious,  Political,  Commercial, 
Tribe,  Clan,  Family,  Criminal 

•  Location :  Geo-spatial  Representation,  Military  Grid  Reference  System,  Fatitude/Fongitude 

•  Time :  Temporal  Expression,  Temporal  Entity 

•  Event/Transaction:  Organize,  Plan,  Meet,  Execute,  Communicate,  Conflict,  Transact 

•  Equipment :  IED  Components,  Weapons  Caches,  Communications,  Vehicles 

Deception  was  not  a  specifically  designed  into  the  SYNCOIN  dataset.  However,  misdirection,  falsehoods, 
inaccuracies  and  bad  information  is  resident  in  many  of  the  reports,  interviews,  sightings  and  depictions 
that  form  the  basis  of  the  messages  that  make  up  the  dataset.  This  was  accomplished  to  provide 
background  noise  and  to  give  a  realistic  feel  to  the  messages. 
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SYNCOIN,  while  very  useful,  is  admittedly  imperfect.  While  it  does  some  things  fairly  well,  like 
represent  unstructured  observations  and  semi-structured  reports;  it  falls  short  when  addressing  the 
underlying  specifics  of  hard  sensor  modalities.  This  is  largely  a  design  issue.  SYNCOIN  was  designed 
and  constructed  to  represent  a  mix  of  reports,  observations  and  records  that  might  be  received  and 
recorded  within  a  battalion  level  command  post.  Raw  sensor  data  would  not  likely  be  part  of  this 
exchange,  whereas  an  analysis  of  the  raw  data  would  more  the  norm.  In  fact  it  is  reasonable  that  several 
levels  of  interpretation  might  occur. 

For  example,  the  signal  intercept  of  a  cell  phone  communication  between  a  HVT  and  a  named  person  of 
interest  is  captured  and  recorded  at  the  signals  battalion.  The  record  is  forwarded  to  a  linguist  for 
transcription  and  another  record  is  made.  This  record  is  then  sent  onward  to  a  HUMINT  analyst  for 
interpretation,  and  again  a  summary  record  is  created.  By  this  time,  the  original  signals  intercept  is  only 
reflected  in  the  metadata  (time,  data,  location,  time  of  call,  phone  numbers,  etc.)  that  is  carried  forward 
intact  with  expanding  analysis;  the  raw  signals  are  no  longer  of  interest  at  this  level.  The  HUMINT 
analytic  summary  is  forwarded  to  an  operations  analyst  who  recognizes  the  operational  significance  of  the 
transaction  and  parties  involved  and  the  report  is  further  annotated.  The  annotated  report  is  finally 
forwarded  to  the  command  post  for  information  or  action. 

In  essence,  the  hard-sensor  related  messages  within  SYNCOIN  represent  what  might  be  referred  to  as 
hybrid  reports.  Hall,  et  al  addresses  the  changing  role  of  humans  in  the  data  fusion  process;  whereas 
traditional  data  fusion  systems  focused  primarily  on  fusing  data  from  physical  sensors  to  address  physical 
targets;  the  emerging  trend  focuses  on  human-centered  data  fusion  processes  [18].  The  human-centered 
fusion  approach  has  many  advantages  of  over  traditional  methods  which  rely  on  highly  calibrated  systems 
collecting  data  within  a  specified  domain.  Humans  acting  as  soft  sensors,  have  the  ability  to  detect 
intricacies  of  human  behavior,  mood  and  motivation. 

Future  attempts  in  building  hard/soft  datasets  will  need  to  address  the  handling  and  representation  of 
underlying  hard  sensor  information.  Another  approach  might  be  to  build  up  a  separate  but  corresponding 
hard-data  set  to  augment  SYNCOIN  or  other  existing  datasets  such  as  those  that  have  been  created  for  the 
Analytic  Decision  Game  series  of  exercises. 

While  the  topic  of  SYNCOIN  was  centered  on  the  cause  and  effects  of  IED  networks,  other  equally 
challenging  operational  environments  can  be  addressed  using  scenarios  like  those  that  make  up  the 
analytic  decision  game.  As  a  practical  matter,  any  given  ADG  dataset  is  extensible  beyond  the  exercise 
environment  for  use  in  test  and  evaluation  of  data  fusion  processes. 

1.2  Role  of  the  ADG  in  the  classroom 

The  original  motivation  and  focus  of  the  ADG  series  exercises  and  data  was  to  teach  and  promote  the  use 
of  structured  analytic  techniques.  Richards  Heuer  and  Randolph  Pherson  in  their  outstanding  primer, 
Structured  Analytic  Techniques  (SAT)  for  Intelligence  Analysts  have  set  the  standard  for  teaching 
structured  analytics.  The  SAT  primer  is  the  perfect  companion  for  the  analytic  decision  game. 
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The  adaptability  of  the  ADG  enables  it  to  meet  a  broad  range  of  teaching  and  training  objectives;  it  can  be 
tailored  to  specific  course  content,  terminal  or  enabling  learning  objective  or  constructed  to  act  as  a  stand¬ 
alone  exercise.  The  ADG  offers  an  intensive  experience  in  critical  analysis  by  broadening  students’ 
perspectives  beyond  their  culture  or  educational  discipline.  It  offers  an  opportunity  for  the  introduction  of 
new  analytic  techniques  as  well  as  the  integration  of  knowledge,  skills  and  abilities  (KSAs)  developed  in 
previous  courses  of  study.  The  ADG  requires  active  engagement.  Participants  cannot  just  ride  the  exercise 
wave  as  passive  observers;  the  ADG  requires  students  to  examine  complex  issues  in  substantial  pieces  of 
analysis,  presentation  and  writing. 

In  the  College  of  Information  Sciences  and  Technology,  the  topics  of  the  ADG  address  the  key  disciplines 
of  the  three  (3)  options  that  comprise  the  Security  and  Risk  Analysis  curriculum  -  Social  Factors, 
Information  &  Cyber  Security,  and  Intelligence  Analysis  &  Modeling.  The  analysis  derived  from  a  given 
ADG  scenario  requires  the  application  of  analytic  processes  across  all  three  disciplinary  lines  (SRA 
course  options)  to  examine  a  significant  problem  of  security  and  risk. 

For  example,  the  exercise  written  for  ADG  #  1,  “ Show  me  the  Money,”  focuses  on  funding  global 
terrorism  [20].  Exercise  participants  working  in  teams  must  chart  the  flow  of  money  and  identify  the 
individuals  and  groups  that  comprise  the  financial  network  funding  a  terrorist  organization.  ADG  #1  was 
designed  to  illustrate  the  distributed  nature  of  terrorist  organizations  and  operations  on  a  global  scale. 
Student  analysts,  playing  roles  within  a  Joint  Counter-Terrorism  Task  Force,  are  required  to  piece 
together  the  clues  to  thwart  a  terrorist  plot  while  constructing  visualizations  to  represent  the  terror  group's 
organizational  structure,  area  of  operations,  and  the  operational  continuum  of  the  terror  plot.  Data 
supporting  this  scenario  takes  the  form  of  both  finished  and  raw  intelligence  reports  from  the  various 
agencies  involved. 

ADG  #2  “ Diabolical  Deeds  in  the  District ”  is  similar  from  a  team  analytic  perspective,  but  is  constrained 
in  the  sense  that  analytic  teams  assume  a  single  agency  focus  [21].  Student  analysts  are  formed  in  agency 
teams  to  view  information  and  perform  sense-making  from  a  particular  mission  focus  to  demonstrate  and 
reinforce  what  Okerbloom  calls  "the  influence  of  organizational  culture  on  decision-making."  [22].  In 
ADG  #  2  four  agency  teams  resolve  to  uncover  the  connections  between  a  diamond  heist,  a  terror 
organization  and  a  crash  on  the  Washington,  D.C.  Metro.  Staying  in  their  respective  investigative  lane  is 
each  team’s  challenge.  Data  for  this  scenario  is  represented  by  informal  information  exchanges  between 
agency  members  in  the  form  of  e-mails,  memorandums  and  text  messages. 

For  ADG  #  3,  Piracy  on  the  High  Seas,  the  focus  is  on  Somali  Piracy  [11].  This  scenario  was  expressly 
built  to  support  the  very  first  Security  and  Risk  Analysis  (SRA)  Capstone  course.  SRA  is  a  relatively  new 
major  in  Penn  State’s  college  of  Information  Sciences  and  Technology  (1ST).  Because  this  is  the  capstone 
course,  it  requires  students  to  dedicate  a  large  portion  of  the  course  in  research  and  writing.  Preceding  the 
exercise  student  teams  develop  a  comprehensive  study  of  the  current  state  of  Somali  Piracy.  The  scenario 
written  for  the  course  seeks  to  exercise  one  probable  future  state.  The  High  Seas  scenario  places  students 
in  the  Intelligence  Section  (C-2)  of  a  Combined  Task  Force  (CTF).  Data  supporting  this  scenario  has  the 
look  and  feel  of  intelligence  messages  that  could  be  received  in  a  CTF  command  post.  The  scenario  deals 
with  more  than  just  piracy  however,  causing  students  to  sort  through  the  ambiguities  of  media  and 
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international  political  commentary  to  understand  what  groups  pose  the  real  danger;  all  while  the  threat  of 
piracy  moves  from  the  Gulf  of  Aden  to  tourist  areas  closer  to  home. 

ADG  #  4,  High  Finance,  which  is  under  construction,  will  focus  on  persistent  cyber  threats  to  the 
financial  sector.  The  analytic  agency  of  record  in  this  scenario  is  the  Financial  Services  Information 
Sharing  and  Analysis  Center  (FS-ISAC),  whose  role  is  to  support  the  U.S.  Department  of  Treasury  and 
the  Financial  Services  Sector.  Data  for  this  scenario  emulates  that  disseminated  in  the  Department  of 
Homeland  Security  Daily  Report  (DHS  Daily).  Exercise  participants  are  required  to  make  weekly  cyber 
threat  assessments  and  security  recommendations  to  the  financial  sector  to  balance  their  control  measures 
accordingly.  This  ADG  is  also  being  developed  to  support  the  SRA  Capstone. 

Finally,  ADG  #  5,  Cyber  Futures,  which  is  also  under  construction  and  slated  for  a  future  SRA  Capstone 
exercise,  deal  with  cyber  Armageddon.  The  research  and  writing  assignment  for  this  problem  is  to  derive 
three  probable  cyber  futures  all  in  the  year  2025:  “Life  is  Good,”  where  cyber-space  is  everything  we  ever 
imagined;  “ Status  Quo,”  in  which  we  have  maintained  our  current  cyber  trends;  and  “On  the  Brink,”  in 
which  we  are  facing  cyber  collapse.  The  exercise  supporting  this  capstone  involves  a  “post-cyber 
calamity.”  Exercise  participants  in  this  scenario  take  on  a  very  different  role  in  than  their  predecessors. 
Participants  in  ADG  #  5  assume  the  various  roles  of  community  leaders  who  must  work  together  to 
recover  from  cyber  collapse. 

The  immersive  nature  of  the  exercise  environment  shifts  the  burden  away  from  the  instructor  and  onto  the 
student  participant.  The  goal  is  for  student  participants  to  analyze  their  way  into  and  out  of  the  scenario, 
perform  analysis  from  disparate  data  and  offer  actionable  recommendations  based  on  sound  analytic 
principles. 

The  use  of  simulations  in  the  undergraduate  classroom  is  not  without  its  challenges.  Exercise  design, 
scenario  and  data  creation,  and  exercise  facilitation  all  provide  significant  barriers  to  the  uninitiated. 
Exercise  design  and  data  creation  to  support  an  ADG  simulation  is  an  arduous  task  that  must  be 
thoroughly  researched  and  artfully  represented.  First  the  data  should  sound  and  feel  authentic.  Second,  it 
must  be  presented  in  a  realistic,  if  not  entertaining  fashion  in  order  to  gain  and  hold  the  attention  of  the 
participants.  Today’s  student  who  has  been  weaned  on  high  fidelity  gaming  computers,  software  and 
game  scenarios  want  realism  and  believability.  Even  though  the  simulation  is  built  of  synthetic  data,  it 
must  have  the  look  and  feel  of  the  real  thing;  lest  the  participant  venture  outside  the  scenario  (bringing  in 
external  sources)  in  order  to  challenge  its  veracity. 

Nothing  will  bring  a  scenario  down  harder  or  faster  than  Wikipedia  [23];  therefore  one  word  of  caution  is 
included  in  every  exercise  pre-brief;  “All  text-based  evidence  will  be  provided  by  the  instructor. 
Knowledge  exploration  beyond  the  message  traffic  is  encouraged  to  a  degree.  For  example,  the  internet 
may  be  utilized  to  expand  situational  awareness  (SA)  of  the  named  areas  of  interest  and  is  a  good  source 
for  maps,  photos  or  other  geo-reference  material;  however,  do  not  introduce  written  evidence  from 
outside  the  problem”  [21], 
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Most  undergraduate  students  have  little  experience  outside  the  classroom  and  therefore  may  need 
substantial  coaching  and  prompting  to  get  them  to  think  and  act  the  part  of  the  law  enforcement,  military 
or  industry  analyst  whose  role  they  represent  within  the  scenario.  To  address  this  shortfall  and  to 
effectively  facilitate  the  exercise  in  general  requires  a  modicum  of  knowledge  and/or  experience  in  the 
problem  space  being  represented  in  the  simulation.  This  may  be  a  reason  why  many  traditional  faculty 
members  shy  away  from  the  construction  and  use  of  ADG-type  exercises  in  the  classroom. 
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